Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics
Download as PPT, PDF•0 likes•43 views
The document discusses the development of a community cyberinfrastructure to enhance marine microbial ecology metagenomics, focusing on the challenges of data throughput and the integration of advanced optical networking technologies. It highlights collaborative efforts among various academic institutions and organizations to build high-performance computing resources, such as the Optiputer project, and create a network for data sharing and analysis in metagenomics research. Key initiatives include the marine genome sequencing project and the establishment of data portals and services to facilitate access to extensive microbial datasets.
Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics
- 1. Building a Community Cyberinfrastructure to Support Marine Microbial Ecology Metagenomics Invited Talk Metagenomics 2006 Calit2 @ UCSD La Jolla, CA October 4, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
- 2. Challenge: Average Throughput of NASA Data Products to End User is < 50 Mbps Tested October 2005 http://ensight.eos.nasa.gov/Missions/icesat/index.shtml Internet2 Backbone is 10,000 Mbps! Throughput is < 0.5% to End User
- 3. Dedicated Optical Channels Makes High Performance Cyberinfrastructure Possible Parallel Lambdas are Driving Optical Networking The Way Parallel Processors Drove 1990s Computing ( WDM) Source: Steve Wallach, Chiaro Networks “ Lambdas”
- 4. National Lambda Rail (NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers San Francisco Pittsburgh Cleveland San Diego Los Angeles Portland Seattle Pensacola Baton Rouge Houston San Antonio Las Cruces / El Paso Phoenix New York City Washington, DC Raleigh Jacksonville Dallas Tulsa Atlanta Kansas City Denver Ogden/ Salt Lake City Boise Albuquerque UC-TeraGrid UIC/NW-Starlight Chicago International Collaborators NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone Links Two Dozen State and Regional Optical Networks DOE, NSF, & NASA Using NLR
- 5. The OptIPuter Project – Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data NSF Large Information Technology Research Proposal Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI Partnering Campuses: SDSC, USC, SDSU, NCSA, NW, TA&M, UvA, SARA, NASA Goddard, KISTI, AIST, CRC(Canada), CICESE (Mexico) Engaged Industrial Partners: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5 Million Over Five Years—Now In the Fifth Year NIH Biomedical Informatics Research Network NSF EarthScope and ORION
- 6. OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid GTP XCP UDT LambdaStream CEP RBUDP Globus XIO GRAM GSI Source: Andrew Chien, UCSD DVC Configuration Distributed Virtual Computer (DVC) API DVC Runtime Library Distributed Applications/ Web Services Telescience Vol-a-Tile SAGE JuxtaView Visualization Data Services LambdaRAM DVC Services DVC Core Services DVC Job Scheduling DVC Communication Resource Identify/Acquire Namespace Management Security Management High Speed Communication Storage Services IP Lambdas Discovery and Control PIN/PDC RobuStore
- 7. Calit2 “Lives in the Future” By Building Systems of Emerging Disruptive Technologies Co-Evolution of Personal Automobile and Highway/Petroleum Infrastructure Source: Harry Dent, The Great Boom Ahead Technologies Diffuse Into Society Following an S-Curve Calit2 Works Here {
- 8. Calit2--A Systems Approach to the Future of the Internet and its Transformation of Our Society www.calit2.net Calit2 Has Assembled a Complex Social Network of Over 350 UC San Diego & UC Irvine Faculty Working in Multidisciplinary Teams With Staff, Students, Industry, and the Community Integrating Technology Consumers and Producers Into “Living Laboratories”
- 9. Calit2 Brings Computer Scientists and Engineers Together with Biomedical Researchers Some Areas of Concentration: Metagenomics Genomic Analysis of Organisms Evolution of Genomes Cancer Genomics Human Genomic Variation and Disease Proteomics Mitochondrial Evolution Computational Biology Information Theory and Biological Systems UC San Diego UC Irvine
- 10. Evolution is the Principle of Biological Systems: Most of Evolutionary Time Was in the Microbial World Source: Carl Woese, et al You Are Here Much of Genome Work Has Occurred in Animals
- 11. PI Larry Smarr Paul Gilna Ex. Dir. Calit2 is Now Attracting Private Foundation Grants Announced January 17, 2006--$24.5M Over Seven Years
- 12. Marine Genome Sequencing Project – Measuring the Genetic Diversity of Ocean Microbes Sorcerer II Data Will Double Number of Proteins in GenBank!
- 13. Current Universe of Medium/ Large Protein Families Source: Shibu Yooseph, et al. (PLOS Biology in press 2006) Protein Families Conserved Across Tree of Life Protein Families Unique to GOS 17,067 Protein Family Clusters
- 15. Calit2’s Direct Access Core Architecture Will Create Next Generation Metagenomics Server Traditional User Response Request Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Sargasso Sea Data Sorcerer II Expedition (GOS) JGI Community Sequencing Project Moore Marine Microbial Project NASA Goddard Satellite Data Community Microbial Metagenomics Data Flat File Server Farm W E B PORTAL Dedicated Compute Farm (1000 CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs) Data- Base Farm 10 GigE Fabric Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns
- 16. The Bioinformatics Core of the Joint Center for Structural Genomics will be Housed in the Calit2@UCSD Building Extremely Thermostable -- Useful for Many Industrial Processes (e.g. Chemical and Food) 173 Structures (122 from JCSG) Determining the Protein Structures of the Thermotoga Maritima Genome 122 T.M. Structures Solved by JCSG (75 Unique In The PDB) Direct Structural Coverage of 25% of the Expressed Soluble Proteins Probably Represents the Highest Structural Coverage of Any Organism Source: John Wooley, UCSD
- 17. Interactive Visualization of Thermatoga Proteins at Calit2 Source: John Wooley, Jurgen Schulze, Calit2
- 18. OptIPortal– Termination Device for the OptIPuter Global Backplane 20 Dual CPU Nodes, 20 24” Monitors, ~$50,000 1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC! Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC Source: Phil Papadopoulos SDSC, Calit2
- 19. Calit2 is Now OptIPuter Connecting Remote Moore-Funded Microbial Researchers NW! CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals OptIPortal
- 20. Calit2 and the Venter Institute Will Combine Telepresence with Remote Interactive Analysis Live Demonstration of 21st Century National-Scale Team Science OptIPuter Visualized Data HDTV Over Lambda 25 Miles Venter Institute
- 21. Countries are Aggressively Creating Gigabit Services: Interactive Access to CAMERA and LOOKING Systems www.glif.is Created in Reykjavik, Iceland 2003 Visualization courtesy of Bob Patterson, NCSA.
- 22. New OptIPuter Driver: Gigabit Fibers on the Ocean Floor -- Controlling Sensors and HDTV Cameras Remotely National Science Foundation Is Planning a New Generation of Ocean Observatories Ocean Research Interactive Observatory Networks (ORION) Fibered Observatories Linked to Land Fiber Infrastructure Laboratory for the Ocean Observatory Knowledge Integration Grid ( LOOKING ) Building a Prototype Based on OptIPuter Technologies Plus Web/Grid Services HDTV Streams Over IP Will be a Major Driver (Funded by NSF ITR- John Delaney, UWash, PI) LOOKING is Driven By NEPTUNE CI Requirements Making Management of Gigabit Flows Routine
- 23. Using the OptIPuter to Couple Data Assimilation Models to Remote Data Sources Including Biology Regional Ocean Modeling System (ROMS) http://ourocean.jpl.nasa.gov/ NASA MODIS Mean Primary Productivity for April 2001 in California Current System
- 24. Deploying Novel Infrastructure Enables New Science: Gigabit Fibers on the Ocean Floor Source: John Delaney & Deborah Kelley, UWash Canadian-U.S. Collaboration An Experiment in the NSF Laboratory for the Ocean Observatory Knowledge Integration Grid (LOOKING) ITR Prototype of CI for NSF’s ORION
- 25. High Definition Still Frame of Hydrothermal Vent Ecology 2.3 Km Deep White Filamentous Bacteria on 'Pill Bug' Outer Carapace Source: John Delaney and Research Channel, U Washington 1 cm.
- 26. A Near Future Metagenomics Fiber Optic-Enabled Data Generator Source John Delaney, UWash